Rows: 93 Columns: 981── Column specification ────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (34): gvkey, indfmt, consol, popsrc, datafmt, tic, cusip, conm, acctchg, acctstd, acqmeth, compst, curcd, curncd, final, cik, ...
dbl (436): fyear, ajex, ajp, currtr, fyr, ismod, ltcm, pddur, scf, src, upd, acchg, acdo, aco, acodo, acominc, acox, acqao, acqcshi...
lgl (506): adrr, bspr, curuscn, ogm, stalt, udpl, acco, accrt, acoxar, acqlntal, acqniintc, adpac, aedi, afudcc, afudci, amc, amdc,...
date (5): datadate, apdedate, fdate, pdate, ipodate
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Bar chart
Create a bar chart of Meta (Facebook) profits by fiscal year
d1 %>%
filter(tic == "META") %>%
hchart(type = "column" ,hcaes(x = fyear, y = oibdp)) %>%
hc_add_theme(hc_theme_economist())
Play around with a few themes and then finalize one theme:
https://jkunst.com/highcharter/articles/themes.html#themes-1
https://jkunst.com/highcharter/reference/index.html#section-themes
When you hover mouse over the bars, you will notice the tooltip shows
“Series 1” rather than the variable name oibdp. Admittedly,
both “Series 1” and oibdp are equally obtuse but we can do
better. Let’s label the series “Operating Profits”
d1 %>%
filter(tic == "META") %>%
hchart("column" ,hcaes(x = fyear, y = oibdp), name = "Operating Income") %>%
hc_add_theme(hc_theme_monokai())
When you hover mouse over the bars, you will notice the tooltip shows
“Series 1” rather than the variable name oibdp. Admittedly,
both “Series 1” and oibdp are equally obtuse but we can do
better. Let’s label the series “Operating Profits”
d1 %>%
filter(tic == "META") %>%
hchart("column" ,hcaes(x = fyear, y = oibdp)) %>%
hc_tooltip(
headerFormat = "<b>Fiscal Year: {point.key}</b> <br>",
pointFormat = "<b>Operating Profit: {point.y}</b>") %>%
hc_add_theme(hc_theme_monokai())
Change the axes titles
d1 %>%
filter(tic == "META") %>%
hchart("column" ,hcaes(x = fyear, y = oibdp)) %>%
hc_tooltip(
headerFormat = "<b>Fiscal Year: {point.key}</b> <br>",
pointFormat = "<b>Operating Profit: {point.y}</b>") %>%
hc_xAxis(title = list(text = "Fiscal Year",
style = list(color = "#000000", fontWeight = "bold"))) %>%
hc_yAxis(title = list(text = "Operating Income in Million USD",
style = list(color = "#000000", fontWeight = "bold"))) %>%
hc_add_theme(hc_theme_ft())
Add title and subtitle
d1 %>%
filter(tic == "META") %>%
hchart("column" ,hcaes(x = fyear, y = oibdp)) %>%
hc_tooltip(
headerFormat = "<b>Fiscal Year: {point.key}</b> <br>",
pointFormat = "<b>Operating Profit: {point.y}</b>") %>%
hc_xAxis(title = list(text = "Fiscal Year")) %>%
hc_yAxis(title = list(text = "Operating Income in Million USD")) %>%
hc_title(text = "Evil = Profitable??", align = "center") %>%
hc_subtitle(text = "Despite all the scandals, Facebook's operating income is consistently growing",
align = "center") %>%
hc_add_theme(hc_theme_monokai())
Add an emoji to the title. This requires useHTML set to
TRUE inside hc_title(). You can get a relevant
decimal code for any emoji online. For example, check this out: https://www.w3schools.com/charsets/ref_emoji.asp
d1 %>%
filter(tic == "META") %>%
hchart("column" ,hcaes(x = fyear, y = oibdp)) %>%
hc_tooltip(
headerFormat = "<b>Fiscal Year: {point.key}</b> <br>",
pointFormat = "<b>Operating Profit: {point.y}</b>") %>%
hc_xAxis(title = list(text = "Fiscal Year")) %>%
hc_yAxis(title = list(text = "Operating Income in Million USD")) %>%
hc_title(text = '<span style="font-family: helvetica, arial, sans-serif;"><strong><span style="color: #3598db;">Evil = Profitable??</span></strong></span> <span>😈</span>',
useHTML = TRUE, align = "center") %>%
hc_subtitle(text = "Despite all the scandals, Facebook operating income is consistently growing",
align = "center") %>%
hc_add_theme(hc_theme_monokai())
Line graph
Plot a line graph of Apple’s leverage ratio over the years.
The leverage ratio is given by (dltt + dlc) / at where
dltt is long-term debt and dlc is short-term
debt. If any of these variables is missing, we should assume they are
0.
d1 %>%
filter(tic == "AAPL") %>%
mutate(leverage = (replace_na(dltt, 0) + replace_na(dlc, 0))/ at,
leverage = round(leverage, 2)) %>%
hchart("line", hcaes(x = fyear, y = leverage), name = "Leverage") %>%
hc_add_theme(hc_theme_bloom())
Scatterplot
Create a scatterplot of profit by sales
d1 %>%
hchart("scatter", hcaes(x = sale, y = oibdp, group = conm)) %>%
hc_add_theme(hc_theme_538())
Here, the colors are repeating because the color palette doesn’t have
7 unique colors. We can pass our own colors using
hc_colors(). I will use viridis color palettes
because it has 256 colors.
Notice that cols has 8 characters instead of 6 as expected in the hex
code. The two extra characters are for alpha. We have to remove them
before we can use them here. Read more about it here: https://www.quackit.com/css/color/values/css_hex_color_notation_8_digits.cfm
cols <- viridisLite::viridis(10, option = "H")
cols <- substr(cols, 0, 7) # This will retain first 7 characters including the # sign.
d1 %>%
hchart("scatter", hcaes(x = sale, y = oibdp, group = conm)) %>%
hc_add_theme(hc_theme_538()) %>%
hc_colors(cols)
Use pals package for more discrete colors: https://rdrr.io/cran/pals/man/discrete.html
cols <- pals::alphabet() %>% unname()
# unname will remove the names from the vector. Highcharter throws an error if you pass a names vector with colors.
d1 %>%
hchart("scatter", hcaes(x = sale, y = oibdp, group = conm)) %>%
hc_add_theme(hc_theme_538()) %>%
hc_colors(cols)
Adding a regression line to the scatterplot
We will use a regression plugin to get a regression line. The plugins
are available in the following folder on your computer:
dir(system.file(“htmlwidgets/lib/highcharts/plugins”, package =
“highcharter”))
d1 %>%
filter(tic == "AAPL") %>%
hchart("scatter", hcaes(x = sale, y = oibdp), regression = TRUE) %>%
hc_add_dependency("plugins/highcharts-regression.js") %>%
hc_add_theme(hc_theme_538())
Heatmap
We did not see static heatmaps with ggplot2 so let’s
learn how to get them using highcharter
To create a heatmap, highcharter needs a data frame or a matrix. A
matrix is a data object similar to a data frame as it is 2D. However, a
matrix must have ALL its elements of the same class. This is different
from a data frame where every column must have elements of the same
class but two columns may have different classes.
The most common heatmap is a correlation plot. cor()
function from base R outputs a matrix of correlations. Here we visualize
correlations between multiple variables in the data set.
cor_dt = d1 %>%
select(sale, oibdp, cogs, at, xrd, mkvalt, che, capx) %>%
drop_na() %>%
cor()
cor_dt %>%
hchart() %>%
hc_colorAxis(
stops = color_stops(colors = rev(c("#000004FF",
"#56106EFF",
"#BB3754FF",
"#F98C0AFF",
"#FCFFA4FF")))
)
Why are all the correlations positive? For example, why is the
correlation between profits (oibdp) and the costs
(cogs) positive? Shouldn’t that be negative?
d2 <- d1 %>%
select(sale, oibdp, cogs, at, xrd, mkvalt, che, capx) %>%
drop_na()
d2 %>%
mutate(across(everything(), ~.x/sale)) %>%
select(-sale) %>%
cor() %>%
hchart()
JB has given an example of heatmap using a data frame here: https://jkunst.com/highcharter/articles/highcharter.html
Pie chart
Let’s create a pie chart for all the profits in the fiscal year 2021.
This will be an ugly pie chart because there are 10 companies in the
data. But interactive pie charts make life a little bit easier!
d1 %>%
filter(fyear == 2021) %>%
arrange(oibdp) %>%
mutate(conm = str_to_title(conm)) %>% # Convert company names to title case
hchart("pie", hcaes(x = conm, y = oibdp))
Change the tooltip to show the series name:
d1 %>%
filter(fyear == 2021) %>%
arrange(oibdp) %>%
mutate(conm = str_to_title(conm)) %>%
hchart("pie", hcaes(x = conm, y = oibdp), name = "Operating Profit")
Wordcloud
Highcharter has a module that allows us to create wordclopuds with
minimal effort.
To create the wordcloud, we need words and their frequencies from
text. For this example, I will use a few random tweets collected in
November 2021.
tweets = readRDS(here::here("data", "file001.rds"))
We will use tidytext package for getting the words from the
tweets.
tweet_text = tibble(text = tweets$text) %>%
mutate(text = gsub(x = text, pattern = "[0-9]+|[[:punct:]]|\\(.*\\)", replacement = "")) %>%
tidytext::unnest_tokens(word, text)
Next, let’s delete common words that don’t add much to the analysis.
These are words such as “the”, “and”, etc. and called stopwords.
data(stop_words)
tweet_text = tweet_text %>% anti_join(stop_words)
Joining with `by = join_by(word)`
tweet_text %>%
count(word, sort = TRUE)
We can remove other commonly occurring useless words as follows:
tweet_text = tweet_text %>%
filter(!word %in% c("im", "dont", "amp", "youre", "ive", "hes", "didnt", "isnt"))
Now we are ready for the wordcloud
word_freq = tweet_text %>%
count(word, sort = T)
word_freq %>%
filter(n > 100) %>%
hchart("wordcloud", hcaes(name = word, weight = n), name = "Count")
Wordcloud using wordcloud2 package
word_freq %>%
filter(n > 100) %>%
wordcloud2(color = "random-light", backgroundColor = "black",
minRotation = 0, maxRotation = 0)
Visualizing stock market data
We will compare the stock price movement of Apple and Microsoft
aapl = getSymbols("AAPL", auto.assign = FALSE)
msft = getSymbols("MSFT", auto.assign = FALSE)
highchart(type = "stock") %>%
hc_add_series(aapl) %>%
hc_add_series(msft)
LS0tCnRpdGxlOiAiSGlnaGNoYXJ0ZXIgRXhhbXBsZXMiCm91dHB1dDoKICBodG1sX25vdGVib29rOgogICAgdGhlbWU6IGNvc21vCmVkaXRvcl9vcHRpb25zOgogIGNodW5rX291dHB1dF90eXBlOiBpbmxpbmUKLS0tCgpgYGB7ciBzZXR1cCwgZWNobz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KCnBhY21hbjo6cF9sb2FkKHRpZHl2ZXJzZSwgaGlnaGNoYXJ0ZXIsIHRpZHl0ZXh0LCBxdWFudG1vZCkKcGFjbWFuOjpwX2xvYWRfZ2goImxjaGlmZm9uL3dvcmRjbG91ZDIiKQoKZDEgPC0gcmVhZF9jc3YoaGVyZTo6aGVyZSgiZGF0YSIsICJ0ZWNoX3N0b2Nrc19jc3YuemlwIikpCmBgYAoKIyMgQmFyIGNoYXJ0CgpDcmVhdGUgYSBiYXIgY2hhcnQgb2YgTWV0YSAoRmFjZWJvb2spIHByb2ZpdHMgYnkgZmlzY2FsIHllYXIKCmBgYHtyfQpkMSAlPiUgCiAgZmlsdGVyKHRpYyA9PSAiTUVUQSIpICU+JSAKICBoY2hhcnQodHlwZSA9ICJjb2x1bW4iICxoY2Flcyh4ID0gZnllYXIsIHkgPSBvaWJkcCkpICU+JSAKICBoY19hZGRfdGhlbWUoaGNfdGhlbWVfZWNvbm9taXN0KCkpCmBgYAoKKlBsYXkgYXJvdW5kIHdpdGggYSBmZXcgdGhlbWVzIGFuZCB0aGVuIGZpbmFsaXplIG9uZSB0aGVtZToqIApodHRwczovL2prdW5zdC5jb20vaGlnaGNoYXJ0ZXIvYXJ0aWNsZXMvdGhlbWVzLmh0bWwjdGhlbWVzLTEKaHR0cHM6Ly9qa3Vuc3QuY29tL2hpZ2hjaGFydGVyL3JlZmVyZW5jZS9pbmRleC5odG1sI3NlY3Rpb24tdGhlbWVzCgoKV2hlbiB5b3UgaG92ZXIgbW91c2Ugb3ZlciB0aGUgYmFycywgeW91IHdpbGwgbm90aWNlIHRoZSB0b29sdGlwIHNob3dzICJTZXJpZXMgMSIgcmF0aGVyIHRoYW4gdGhlIHZhcmlhYmxlIG5hbWUgYG9pYmRwYC4gQWRtaXR0ZWRseSwgYm90aCAiU2VyaWVzIDEiIGFuZCBgb2liZHBgIGFyZSBlcXVhbGx5IG9idHVzZSBidXQgd2UgY2FuIGRvIGJldHRlci4gTGV0J3MgbGFiZWwgdGhlIHNlcmllcyAiT3BlcmF0aW5nIFByb2ZpdHMiCgpgYGB7cn0KZDEgJT4lIAogIGZpbHRlcih0aWMgPT0gIk1FVEEiKSAlPiUgCiAgaGNoYXJ0KCJjb2x1bW4iICxoY2Flcyh4ID0gZnllYXIsIHkgPSBvaWJkcCksIG5hbWUgPSAiT3BlcmF0aW5nIEluY29tZSIpICU+JSAKICBoY19hZGRfdGhlbWUoaGNfdGhlbWVfbW9ub2thaSgpKQpgYGAKCldoZW4geW91IGhvdmVyIG1vdXNlIG92ZXIgdGhlIGJhcnMsIHlvdSB3aWxsIG5vdGljZSB0aGUgdG9vbHRpcCBzaG93cyAiU2VyaWVzIDEiIHJhdGhlciB0aGFuIHRoZSB2YXJpYWJsZSBuYW1lIGBvaWJkcGAuIEFkbWl0dGVkbHksIGJvdGggIlNlcmllcyAxIiBhbmQgYG9pYmRwYCBhcmUgZXF1YWxseSBvYnR1c2UgYnV0IHdlIGNhbiBkbyBiZXR0ZXIuIExldCdzIGxhYmVsIHRoZSBzZXJpZXMgIk9wZXJhdGluZyBQcm9maXRzIgoKCmBgYHtyfQpkMSAlPiUgCiAgZmlsdGVyKHRpYyA9PSAiTUVUQSIpICU+JSAKICBoY2hhcnQoImNvbHVtbiIgLGhjYWVzKHggPSBmeWVhciwgeSA9IG9pYmRwKSkgJT4lIAogIGhjX3Rvb2x0aXAoCiAgICBoZWFkZXJGb3JtYXQgPSAiPGI+RmlzY2FsIFllYXI6IHtwb2ludC5rZXl9PC9iPiA8YnI+IiwKICAgIHBvaW50Rm9ybWF0ID0gIjxiPk9wZXJhdGluZyBQcm9maXQ6IHtwb2ludC55fTwvYj4iKSAlPiUgCiAgaGNfYWRkX3RoZW1lKGhjX3RoZW1lX21vbm9rYWkoKSkKYGBgCgpDaGFuZ2UgdGhlIGF4ZXMgdGl0bGVzCgpgYGB7cn0KZDEgJT4lIAogIGZpbHRlcih0aWMgPT0gIk1FVEEiKSAlPiUgCiAgaGNoYXJ0KCJjb2x1bW4iICxoY2Flcyh4ID0gZnllYXIsIHkgPSBvaWJkcCkpICU+JSAKICBoY190b29sdGlwKAogICAgaGVhZGVyRm9ybWF0ID0gIjxiPkZpc2NhbCBZZWFyOiB7cG9pbnQua2V5fTwvYj4gPGJyPiIsCiAgICBwb2ludEZvcm1hdCA9ICI8Yj5PcGVyYXRpbmcgUHJvZml0OiB7cG9pbnQueX08L2I+IikgJT4lIAogIGhjX3hBeGlzKHRpdGxlID0gbGlzdCh0ZXh0ID0gIkZpc2NhbCBZZWFyIiwKICAgICAgICAgICAgICAgICAgICAgICAgc3R5bGUgPSBsaXN0KGNvbG9yID0gIiMwMDAwMDAiLCBmb250V2VpZ2h0ID0gImJvbGQiKSkpICU+JSAKICBoY195QXhpcyh0aXRsZSA9IGxpc3QodGV4dCA9ICJPcGVyYXRpbmcgSW5jb21lIGluIE1pbGxpb24gVVNEIiwKICAgICAgICAgICAgICAgICAgICAgICAgc3R5bGUgPSBsaXN0KGNvbG9yID0gIiMwMDAwMDAiLCBmb250V2VpZ2h0ID0gImJvbGQiKSkpICU+JSAKICBoY19hZGRfdGhlbWUoaGNfdGhlbWVfZnQoKSkKYGBgCgpBZGQgdGl0bGUgYW5kIHN1YnRpdGxlCgpgYGB7cn0KZDEgJT4lIAogIGZpbHRlcih0aWMgPT0gIk1FVEEiKSAlPiUgCiAgaGNoYXJ0KCJjb2x1bW4iICxoY2Flcyh4ID0gZnllYXIsIHkgPSBvaWJkcCkpICU+JSAKICBoY190b29sdGlwKAogICAgaGVhZGVyRm9ybWF0ID0gIjxiPkZpc2NhbCBZZWFyOiB7cG9pbnQua2V5fTwvYj4gPGJyPiIsCiAgICBwb2ludEZvcm1hdCA9ICI8Yj5PcGVyYXRpbmcgUHJvZml0OiB7cG9pbnQueX08L2I+IikgJT4lIAogIGhjX3hBeGlzKHRpdGxlID0gbGlzdCh0ZXh0ID0gIkZpc2NhbCBZZWFyIikpICU+JSAKICBoY195QXhpcyh0aXRsZSA9IGxpc3QodGV4dCA9ICJPcGVyYXRpbmcgSW5jb21lIGluIE1pbGxpb24gVVNEIikpICU+JSAKICBoY190aXRsZSh0ZXh0ID0gIkV2aWwgPSBQcm9maXRhYmxlPz8iLCBhbGlnbiA9ICJjZW50ZXIiKSAlPiUKICBoY19zdWJ0aXRsZSh0ZXh0ID0gIkRlc3BpdGUgYWxsIHRoZSBzY2FuZGFscywgRmFjZWJvb2sncyBvcGVyYXRpbmcgaW5jb21lIGlzIGNvbnNpc3RlbnRseSBncm93aW5nIiwKICAgICAgICAgICAgICBhbGlnbiA9ICJjZW50ZXIiKSAlPiUgCiAgaGNfYWRkX3RoZW1lKGhjX3RoZW1lX21vbm9rYWkoKSkKYGBgCgoKQWRkIGFuIGVtb2ppIHRvIHRoZSB0aXRsZS4gVGhpcyByZXF1aXJlcyBgdXNlSFRNTGAgc2V0IHRvIGBUUlVFYCBpbnNpZGUgYGhjX3RpdGxlKClgLiBZb3UgY2FuIGdldCBhIHJlbGV2YW50IGRlY2ltYWwgY29kZSBmb3IgYW55IGVtb2ppIG9ubGluZS4gRm9yIGV4YW1wbGUsIGNoZWNrIHRoaXMgb3V0OgpodHRwczovL3d3dy53M3NjaG9vbHMuY29tL2NoYXJzZXRzL3JlZl9lbW9qaS5hc3AKCgpgYGB7cn0KZDEgJT4lIAogIGZpbHRlcih0aWMgPT0gIk1FVEEiKSAlPiUgCiAgaGNoYXJ0KCJjb2x1bW4iICxoY2Flcyh4ID0gZnllYXIsIHkgPSBvaWJkcCkpICU+JSAKICBoY190b29sdGlwKAogICAgaGVhZGVyRm9ybWF0ID0gIjxiPkZpc2NhbCBZZWFyOiB7cG9pbnQua2V5fTwvYj4gPGJyPiIsCiAgICBwb2ludEZvcm1hdCA9ICI8Yj5PcGVyYXRpbmcgUHJvZml0OiB7cG9pbnQueX08L2I+IikgJT4lIAogIGhjX3hBeGlzKHRpdGxlID0gbGlzdCh0ZXh0ID0gIkZpc2NhbCBZZWFyIikpICU+JSAKICBoY195QXhpcyh0aXRsZSA9IGxpc3QodGV4dCA9ICJPcGVyYXRpbmcgSW5jb21lIGluIE1pbGxpb24gVVNEIikpICU+JSAKICBoY190aXRsZSh0ZXh0ID0gJzxzcGFuIHN0eWxlPSJmb250LWZhbWlseTogaGVsdmV0aWNhLCBhcmlhbCwgc2Fucy1zZXJpZjsiPjxzdHJvbmc+PHNwYW4gc3R5bGU9ImNvbG9yOiAjMzU5OGRiOyI+RXZpbCA9IFByb2ZpdGFibGU/Pzwvc3Bhbj48L3N0cm9uZz48L3NwYW4+IDxzcGFuPiYjMTI4NTIwOzwvc3Bhbj4nLAogICAgICAgICAgIHVzZUhUTUwgPSBUUlVFLCBhbGlnbiA9ICJjZW50ZXIiKSAlPiUKICBoY19zdWJ0aXRsZSh0ZXh0ID0gIkRlc3BpdGUgYWxsIHRoZSBzY2FuZGFscywgRmFjZWJvb2sgb3BlcmF0aW5nIGluY29tZSBpcyBjb25zaXN0ZW50bHkgZ3Jvd2luZyIsCiAgICAgICAgICAgICAgYWxpZ24gPSAiY2VudGVyIikgJT4lIAogIGhjX2FkZF90aGVtZShoY190aGVtZV9tb25va2FpKCkpCmBgYAoKIyMgTGluZSBncmFwaAoKUGxvdCBhIGxpbmUgZ3JhcGggb2YgQXBwbGUncyBsZXZlcmFnZSByYXRpbyBvdmVyIHRoZSB5ZWFycy4KClRoZSBsZXZlcmFnZSByYXRpbyBpcyBnaXZlbiBieSBgKGRsdHQgKyBkbGMpIC8gYXRgIHdoZXJlIGBkbHR0YCBpcyBsb25nLXRlcm0gZGVidCBhbmQgYGRsY2AgaXMgc2hvcnQtdGVybSBkZWJ0LiBJZiBhbnkgb2YgdGhlc2UgdmFyaWFibGVzIGlzIG1pc3NpbmcsIHdlIHNob3VsZCBhc3N1bWUgdGhleSBhcmUgMC4KCmBgYHtyfQpkMSAlPiUgCiAgZmlsdGVyKHRpYyA9PSAiQUFQTCIpICU+JSAKICBtdXRhdGUobGV2ZXJhZ2UgPSAocmVwbGFjZV9uYShkbHR0LCAwKSArIHJlcGxhY2VfbmEoZGxjLCAwKSkvIGF0LAogICAgICAgICBsZXZlcmFnZSA9IHJvdW5kKGxldmVyYWdlLCAyKSkgJT4lIAogIGhjaGFydCgibGluZSIsIGhjYWVzKHggPSBmeWVhciwgeSA9IGxldmVyYWdlKSwgbmFtZSA9ICJMZXZlcmFnZSIpICU+JSAKICBoY19hZGRfdGhlbWUoaGNfdGhlbWVfYmxvb20oKSkKYGBgCgojIyBTY2F0dGVycGxvdAoKQ3JlYXRlIGEgc2NhdHRlcnBsb3Qgb2YgcHJvZml0IGJ5IHNhbGVzCgpgYGB7cn0KZDEgJT4lIAogIGhjaGFydCgic2NhdHRlciIsIGhjYWVzKHggPSBzYWxlLCB5ID0gb2liZHAsIGdyb3VwID0gY29ubSkpICU+JSAKICBoY19hZGRfdGhlbWUoaGNfdGhlbWVfNTM4KCkpCmBgYAoKSGVyZSwgdGhlIGNvbG9ycyBhcmUgcmVwZWF0aW5nIGJlY2F1c2UgdGhlIGNvbG9yIHBhbGV0dGUgZG9lc24ndCBoYXZlIDcgdW5pcXVlIGNvbG9ycy4gV2UgY2FuIHBhc3Mgb3VyIG93biBjb2xvcnMgdXNpbmcgYGhjX2NvbG9ycygpYC4gSSB3aWxsIHVzZSBgdmlyaWRpc2AgY29sb3IgcGFsZXR0ZXMgYmVjYXVzZSBpdCBoYXMgMjU2IGNvbG9ycy4KCk5vdGljZSB0aGF0IGNvbHMgaGFzIDggY2hhcmFjdGVycyBpbnN0ZWFkIG9mIDYgYXMgZXhwZWN0ZWQgaW4gdGhlIGhleCBjb2RlLiBUaGUgdHdvIGV4dHJhIGNoYXJhY3RlcnMgYXJlIGZvciBhbHBoYS4gV2UgaGF2ZSB0byByZW1vdmUgdGhlbSBiZWZvcmUgd2UgY2FuIHVzZSB0aGVtIGhlcmUuIFJlYWQgbW9yZSBhYm91dCBpdCBoZXJlOiBodHRwczovL3d3dy5xdWFja2l0LmNvbS9jc3MvY29sb3IvdmFsdWVzL2Nzc19oZXhfY29sb3Jfbm90YXRpb25fOF9kaWdpdHMuY2ZtCgpgYGB7cn0KY29scyA8LSB2aXJpZGlzTGl0ZTo6dmlyaWRpcygxMCwgb3B0aW9uID0gIkgiKQoKY29scyA8LSBzdWJzdHIoY29scywgMCwgNykgIyBUaGlzIHdpbGwgcmV0YWluIGZpcnN0IDcgY2hhcmFjdGVycyBpbmNsdWRpbmcgdGhlICMgc2lnbi4KCmQxICU+JSAKICBoY2hhcnQoInNjYXR0ZXIiLCBoY2Flcyh4ID0gc2FsZSwgeSA9IG9pYmRwLCBncm91cCA9IGNvbm0pKSAlPiUgCiAgaGNfYWRkX3RoZW1lKGhjX3RoZW1lXzUzOCgpKSAlPiUgCiAgaGNfY29sb3JzKGNvbHMpCmBgYAoKVXNlIGBwYWxzYCBwYWNrYWdlIGZvciBtb3JlIGRpc2NyZXRlIGNvbG9yczoKaHR0cHM6Ly9yZHJyLmlvL2NyYW4vcGFscy9tYW4vZGlzY3JldGUuaHRtbAoKCmBgYHtyfQpjb2xzIDwtIHBhbHM6OmFscGhhYmV0KCkgJT4lIHVubmFtZSgpCiMgdW5uYW1lIHdpbGwgcmVtb3ZlIHRoZSBuYW1lcyBmcm9tIHRoZSB2ZWN0b3IuIEhpZ2hjaGFydGVyIHRocm93cyBhbiBlcnJvciBpZiB5b3UgcGFzcyBhIG5hbWVkIHZlY3RvciB3aXRoIGNvbG9ycy4KCmQxICU+JSAKICBoY2hhcnQoInNjYXR0ZXIiLCBoY2Flcyh4ID0gc2FsZSwgeSA9IG9pYmRwLCBncm91cCA9IGNvbm0pKSAlPiUgCiAgaGNfYWRkX3RoZW1lKGhjX3RoZW1lXzUzOCgpKSAlPiUgCiAgaGNfY29sb3JzKGNvbHMpCmBgYAoKIyMgQWRkaW5nIGEgcmVncmVzc2lvbiBsaW5lIHRvIHRoZSBzY2F0dGVycGxvdAoKV2Ugd2lsbCB1c2UgYSByZWdyZXNzaW9uIHBsdWdpbiB0byBnZXQgYSByZWdyZXNzaW9uIGxpbmUuIFRoZSBwbHVnaW5zIGFyZSBhdmFpbGFibGUgaW4gdGhlIGZvbGxvd2luZyBmb2xkZXIgb24geW91ciBjb21wdXRlcjoKCmRpcihzeXN0ZW0uZmlsZSgiaHRtbHdpZGdldHMvbGliL2hpZ2hjaGFydHMvcGx1Z2lucyIsIHBhY2thZ2UgPSAiaGlnaGNoYXJ0ZXIiKSkKCgpgYGB7cn0KZDEgJT4lIAogIGZpbHRlcih0aWMgPT0gIkFBUEwiKSAlPiUgCiAgaGNoYXJ0KCJzY2F0dGVyIiwgaGNhZXMoeCA9IHNhbGUsIHkgPSBvaWJkcCksIHJlZ3Jlc3Npb24gPSBUUlVFKSAlPiUKICBoY19hZGRfZGVwZW5kZW5jeSgicGx1Z2lucy9oaWdoY2hhcnRzLXJlZ3Jlc3Npb24uanMiKSAlPiUKICBoY19hZGRfdGhlbWUoaGNfdGhlbWVfNTM4KCkpCmBgYAoKCgojIyBIZWF0bWFwCgpXZSBkaWQgbm90IHNlZSBzdGF0aWMgaGVhdG1hcHMgd2l0aCBgZ2dwbG90MmAgc28gbGV0J3MgbGVhcm4gaG93IHRvIGdldCB0aGVtIHVzaW5nIGhpZ2hjaGFydGVyCgpUbyBjcmVhdGUgYSBoZWF0bWFwLCBoaWdoY2hhcnRlciBuZWVkcyBhIGRhdGEgZnJhbWUgb3IgYSBtYXRyaXguIEEgbWF0cml4IGlzIGEgZGF0YSBvYmplY3Qgc2ltaWxhciB0byBhIGRhdGEgZnJhbWUgYXMgaXQgaXMgMkQuIEhvd2V2ZXIsIGEgbWF0cml4IG11c3QgaGF2ZSBBTEwgaXRzIGVsZW1lbnRzIG9mIHRoZSBzYW1lIGNsYXNzLiBUaGlzIGlzIGRpZmZlcmVudCBmcm9tIGEgZGF0YSBmcmFtZSB3aGVyZSBldmVyeSBjb2x1bW4gbXVzdCBoYXZlIGVsZW1lbnRzIG9mIHRoZSBzYW1lIGNsYXNzIGJ1dCB0d28gY29sdW1ucyBtYXkgaGF2ZSBkaWZmZXJlbnQgY2xhc3Nlcy4KClRoZSBtb3N0IGNvbW1vbiBoZWF0bWFwIGlzIGEgY29ycmVsYXRpb24gcGxvdC4gYGNvcigpYCBmdW5jdGlvbiBmcm9tIGJhc2UgUiBvdXRwdXRzIGEgbWF0cml4IG9mIGNvcnJlbGF0aW9ucy4gSGVyZSB3ZSB2aXN1YWxpemUgY29ycmVsYXRpb25zIGJldHdlZW4gbXVsdGlwbGUgdmFyaWFibGVzIGluIHRoZSBkYXRhIHNldC4KCgpgYGB7cn0KY29yX2R0ID0gZDEgJT4lCiAgc2VsZWN0KHNhbGUsIG9pYmRwLCBjb2dzLCBhdCwgeHJkLCBta3ZhbHQsIGNoZSwgY2FweCkgJT4lIAogIGRyb3BfbmEoKSAlPiUgCiAgY29yKCkKYGBgCgoKYGBge3J9CiBjb3JfZHQgJT4lIAogIGhjaGFydCgpICU+JSAKICAgIGhjX2NvbG9yQXhpcygKICAgIHN0b3BzID0gY29sb3Jfc3RvcHMoY29sb3JzID0gcmV2KGMoIiMwMDAwMDRGRiIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICIjNTYxMDZFRkYiLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiI0JCMzc1NEZGIiwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIiNGOThDMEFGRiIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICIjRkNGRkE0RkYiKSkpCiAgICApCmBgYAoKV2h5IGFyZSBhbGwgdGhlIGNvcnJlbGF0aW9ucyBwb3NpdGl2ZT8gRm9yIGV4YW1wbGUsIHdoeSBpcyB0aGUgY29ycmVsYXRpb24gYmV0d2VlbiBwcm9maXRzIChgb2liZHBgKSBhbmQgdGhlIGNvc3RzIChgY29nc2ApIHBvc2l0aXZlPyBTaG91bGRuJ3QgdGhhdCBiZSBuZWdhdGl2ZT8KCgpgYGB7cn0KZDIgPC0gZDEgJT4lCiAgc2VsZWN0KHNhbGUsIG9pYmRwLCBjb2dzLCBhdCwgeHJkLCBta3ZhbHQsIGNoZSwgY2FweCkgJT4lIAogIGRyb3BfbmEoKQoKZDIgJT4lIAogIG11dGF0ZShhY3Jvc3MoZXZlcnl0aGluZygpLCB+Lngvc2FsZSkpICU+JSAKICBzZWxlY3QoLXNhbGUpICU+JSAKICBjb3IoKSAlPiUKICBoY2hhcnQoKQpgYGAKCkpCIGhhcyBnaXZlbiBhbiBleGFtcGxlIG9mIGhlYXRtYXAgdXNpbmcgYSBkYXRhIGZyYW1lIGhlcmU6Cmh0dHBzOi8vamt1bnN0LmNvbS9oaWdoY2hhcnRlci9hcnRpY2xlcy9oaWdoY2hhcnRlci5odG1sCgojIyBQaWUgY2hhcnQKCkxldCdzIGNyZWF0ZSBhIHBpZSBjaGFydCBmb3IgYWxsIHRoZSBwcm9maXRzIGluIHRoZSBmaXNjYWwgeWVhciAyMDIxLiBUaGlzIHdpbGwgYmUgYW4gdWdseSBwaWUgY2hhcnQgYmVjYXVzZSB0aGVyZSBhcmUgMTAgY29tcGFuaWVzIGluIHRoZSBkYXRhLiBCdXQgaW50ZXJhY3RpdmUgcGllIGNoYXJ0cyBtYWtlIGxpZmUgYSBsaXR0bGUgYml0IGVhc2llciEKCmBgYHtyfQpkMSAlPiUgCiAgZmlsdGVyKGZ5ZWFyID09IDIwMjEpICU+JQogIGFycmFuZ2Uob2liZHApICU+JSAKICBtdXRhdGUoY29ubSA9IHN0cl90b190aXRsZShjb25tKSkgJT4lICMgQ29udmVydCBjb21wYW55IG5hbWVzIHRvIHRpdGxlIGNhc2UKICBoY2hhcnQoInBpZSIsIGhjYWVzKHggPSBjb25tLCB5ID0gb2liZHApKQpgYGAKCkNoYW5nZSB0aGUgdG9vbHRpcCB0byBzaG93IHRoZSBzZXJpZXMgbmFtZToKCmBgYHtyfQpkMSAlPiUgCiAgZmlsdGVyKGZ5ZWFyID09IDIwMjEpICU+JQogIGFycmFuZ2Uob2liZHApICU+JSAKICBtdXRhdGUoY29ubSA9IHN0cl90b190aXRsZShjb25tKSkgJT4lIAogIGhjaGFydCgicGllIiwgaGNhZXMoeCA9IGNvbm0sIHkgPSBvaWJkcCksIG5hbWUgPSAiT3BlcmF0aW5nIFByb2ZpdCIpCmBgYAoKCiMjIFdvcmRjbG91ZAoKSGlnaGNoYXJ0ZXIgaGFzIGEgbW9kdWxlIHRoYXQgYWxsb3dzIHVzIHRvIGNyZWF0ZSB3b3JkY2xvcHVkcyB3aXRoIG1pbmltYWwgZWZmb3J0LgoKVG8gY3JlYXRlIHRoZSB3b3JkY2xvdWQsIHdlIG5lZWQgd29yZHMgYW5kIHRoZWlyIGZyZXF1ZW5jaWVzIGZyb20gdGV4dC4gRm9yIHRoaXMgZXhhbXBsZSwgSSB3aWxsIHVzZSBhIGZldyByYW5kb20gdHdlZXRzIGNvbGxlY3RlZCBpbiBOb3ZlbWJlciAyMDIxLgoKYGBge3J9CnR3ZWV0cyA9IHJlYWRSRFMoaGVyZTo6aGVyZSgiZGF0YSIsICJmaWxlMDAxLnJkcyIpKQpgYGAKCldlIHdpbGwgdXNlIHRpZHl0ZXh0IHBhY2thZ2UgZm9yIGdldHRpbmcgdGhlIHdvcmRzIGZyb20gdGhlIHR3ZWV0cy4KCmBgYHtyfQp0d2VldF90ZXh0ID0gdGliYmxlKHRleHQgPSB0d2VldHMkdGV4dCkgJT4lIAogIG11dGF0ZSh0ZXh0ID0gZ3N1Yih4ID0gdGV4dCwgcGF0dGVybiA9ICJbMC05XSt8W1s6cHVuY3Q6XV18XFwoLipcXCkiLCByZXBsYWNlbWVudCA9ICIiKSkgJT4lIAogIHRpZHl0ZXh0Ojp1bm5lc3RfdG9rZW5zKHdvcmQsIHRleHQpCmBgYAoKTmV4dCwgbGV0J3MgZGVsZXRlIGNvbW1vbiB3b3JkcyB0aGF0IGRvbid0IGFkZCBtdWNoIHRvIHRoZSBhbmFseXNpcy4gVGhlc2UgYXJlIHdvcmRzIHN1Y2ggYXMgInRoZSIsICJhbmQiLCBldGMuIGFuZCBjYWxsZWQgc3RvcHdvcmRzLgoKYGBge3J9CmRhdGEoc3RvcF93b3JkcykKCnR3ZWV0X3RleHQgPSB0d2VldF90ZXh0ICU+JSBhbnRpX2pvaW4oc3RvcF93b3JkcykKCmBgYAoKYGBge3J9CnR3ZWV0X3RleHQgJT4lIAogIGNvdW50KHdvcmQsIHNvcnQgPSBUUlVFKQpgYGAKCldlIGNhbiByZW1vdmUgb3RoZXIgY29tbW9ubHkgb2NjdXJyaW5nIHVzZWxlc3Mgd29yZHMgYXMgZm9sbG93czoKCmBgYHtyfQp0d2VldF90ZXh0ID0gdHdlZXRfdGV4dCAlPiUgCiAgZmlsdGVyKCF3b3JkICVpbiUgYygiaW0iLCAiZG9udCIsICJhbXAiLCAieW91cmUiLCAiaXZlIiwgImhlcyIsICJkaWRudCIsICJpc250IikpCmBgYAoKTm93IHdlIGFyZSByZWFkeSBmb3IgdGhlIHdvcmRjbG91ZAoKYGBge3J9CndvcmRfZnJlcSA9IHR3ZWV0X3RleHQgJT4lIAogIGNvdW50KHdvcmQsIHNvcnQgPSBUKQpgYGAKCgpgYGB7cn0Kd29yZF9mcmVxICU+JSAKICBmaWx0ZXIobiA+IDEwMCkgJT4lIAogIGhjaGFydCgid29yZGNsb3VkIiwgaGNhZXMobmFtZSA9IHdvcmQsIHdlaWdodCA9IG4pLCBuYW1lID0gIkNvdW50IikKYGBgCgpXb3JkY2xvdWQgdXNpbmcgd29yZGNsb3VkMiBwYWNrYWdlCgpgYGB7cn0Kd29yZF9mcmVxICU+JSAKICBmaWx0ZXIobiA+IDEwMCkgJT4lIAogIHdvcmRjbG91ZDIoY29sb3IgPSAicmFuZG9tLWxpZ2h0IiwgYmFja2dyb3VuZENvbG9yID0gImJsYWNrIiwKICAgICAgICAgICAgIG1pblJvdGF0aW9uID0gMCwgbWF4Um90YXRpb24gPSAwKQpgYGAKCiMjIFZpc3VhbGl6aW5nIHN0b2NrIG1hcmtldCBkYXRhCgpXZSB3aWxsIGNvbXBhcmUgdGhlIHN0b2NrIHByaWNlIG1vdmVtZW50IG9mIEFwcGxlIGFuZCBNaWNyb3NvZnQKCmBgYHtyfQphYXBsID0gZ2V0U3ltYm9scygiQUFQTCIsIGF1dG8uYXNzaWduID0gRkFMU0UpCm1zZnQgPSBnZXRTeW1ib2xzKCJNU0ZUIiwgYXV0by5hc3NpZ24gPSBGQUxTRSkKYGBgCgpgYGB7cn0KaGlnaGNoYXJ0KHR5cGUgPSAic3RvY2siKSAlPiUgCiAgaGNfYWRkX3NlcmllcyhhYXBsKSAlPiUgCiAgaGNfYWRkX3Nlcmllcyhtc2Z0KQpgYGAKCgo=